Neural Architecture Search: A Visual Analysis

Neural architecture search (NAS) refers to the use of heuristics optimise topology deep neural networks. NAS algorithms have produced topologies that outperform human-designed ones. However, contrasting alternative methods is difficult. To address this, several tabular benchmarks been proposed exhaustively evaluate all architectures in a given space. We conduct thorough fitness landscape analysis popular tabular, cell-based benchmark. Our results indicate landscapes are multi-modal, but relatively low number local optima, from which it not hard escape. confirm reducing noise estimating performance reduces optima. hypothesise local-search based likely be competitive, we by implementing landscape-aware iterated algorithm can more elaborate evolutionary and reinforcement learning methods.